Tumor Segmentation in Intraoperative Fluorescence Images Based on Transfer Learning and Convolutional Neural Networks.

OBJECTIVE: To propose a transfer learning based method of tumor segmentation in intraoperative fluorescence images, which will assist surgeons to efficiently and accurately identify the boundary of tumors of interest. METHODS: We employed transfer learning and deep convolutional neural networks (DCNNs) for tumor segmentation. Specifically, we first pre-trained four networks on the ImageNet dataset to extract low-level features. Subsequently, we fine-tuned these networks on two fluorescence image datasets (ABFM and DTHP) separately to enhance the segmentation performance of fluorescence images. Finally, we tested the trained models on the DTHL dataset. The performance of this approach was compared and evaluated against DCNNs trained end-to-end and the traditional level-set method. RESULTS: The transfer learning-based UNet++ model achieved high segmentation accuracies of 82.17% on the ABFM dataset, 95.61% on the DTHP dataset, and 85.49% on the DTHL test set. For the DTHP dataset, the pre-trained Deeplab v3 + network performed exceptionally well, with a segmentation accuracy of 96.48%. Furthermore, all models achieved segmentation accuracies of over 90% when dealing with the DTHP dataset. CONCLUSION: To the best of our knowledge, this study explores tumor segmentation on intraoperative fluorescent images for the first time. The results show that compared to traditional methods, deep learning has significant advantages in improving segmentation performance. Transfer learning enables deep learning models to perform better on small-sample fluorescence image data compared to end-to-end training. This discovery provides strong support for surgeons to obtain more reliable and accurate image segmentation results during surgery.

Reflective optical imaging for scattering medium using chaotic laser.

Significance: Optical imaging is a non-invasive imaging technology that utilizes near-infrared light, allows for the image reconstruction of optical properties like diffuse and absorption coefficients within the tissue. A recent trend is to use signal processing techniques or new light sources and expanding its application. Aim: We aim to develop the reflective optical imaging using the chaotic correlation technology with chaotic laser and optimize the quality and spatial resolution of reflective optical imaging. Approach: Scattering medium was measured using reflective configuration in different inhomogeneous regions to evaluate the performance of the imaging system. The accuracy of the recovered optical properties was investigated. The reconstruction errors of absorption coefficients and geometric centers were analyzed, and the feature metrics of the reconstructed images were evaluated. Results: We showed how chaotic correlation technology can be utilized for information extraction and image reconstruction. This means that a higher signal-to-noise ratio and image reconstruction of inhomogeneous phantoms under different scenarios successfully were achieved. Conclusions: This work highlights that the peak values of correlation of chaotic exhibit smaller reconstruction error and better reconstruction performance in optical imaging compared with reflective optical imaging with the continuous wave laser.

Biomimetic NIR-II fluorescent proteins created from chemogenic protein-seeking dyes for multicolor deep-tissue bioimaging.

Near-infrared-I/II fluorescent proteins (NIR-I/II FPs) are crucial for in vivo imaging, yet the current NIR-I/II FPs face challenges including scarcity, the requirement for chromophore maturation, and limited emission wavelengths (typically < 800 nm). Here, we utilize synthetic protein-seeking NIR-II dyes as chromophores, which covalently bind to tag proteins (e.g., human serum albumin, HSA) through a site-specific nucleophilic substitution reaction, thereby creating proof-of-concept biomimetic NIR-II FPs. This chemogenic protein-seeking strategy can be accomplished under gentle physiological conditions without catalysis. Proteomics analysis identifies specific binding site (Cys 477 on DIII). NIR-II FPs significantly enhance chromophore brightness and photostability, while improving biocompatibility, allowing for high-performance NIR-II lymphography and angiography. This strategy is universal and applicable in creating a wide range of spectrally separated NIR-I/II FPs for real-time visualization of multiple biological events. Overall, this straightforward biomimetic approach holds the potential to transform fluorescent protein-based bioimaging and enables in-situ albumin targeting to create NIR-I/II FPs for deep-tissue imaging in live organisms.

Evaluation of calibrated and uncalibrated optical imaging approaches for relative cerebral oxygen metabolism measurements in awake mice.

Objective. The continuous delivery of oxygen is critical to sustain brain function, and therefore, measuring brain oxygen consumption can provide vital physiological insight. In this work, we examine the impact of calibration and cerebral blood flow (CBF) measurements on the computation of the relative changes in the cerebral metabolic rate of oxygen consumption (rCMRO2) from hemoglobin-sensitive intrinsic optical imaging data. Using these data, we calculate rCMRO2, and calibrate the model using an isometabolic stimulus.Approach. We used awake head-fixed rodents to obtain hemoglobin-sensitive optical imaging data to test different calibrated and uncalibrated rCMRO2models. Hypercapnia was used for calibration and whisker stimulation was used to test the impact of calibration.Main results. We found that typical uncalibrated models can provide reasonable estimates of rCMRO2with differences as small as 7%-9% compared to their calibrated models. However, calibrated models showed lower variability and less dependence on baseline hemoglobin concentrations. Lastly, we found that supplying the model with measurements of CBF significantly reduced error and variability in rCMRO2change calculations.Significance. The effect of calibration on rCMRO2calculations remains understudied, and we systematically evaluated different rCMRO2calculation scenarios that consider including different measurement combinations. This study provides a quantitative comparison of these scenarios to evaluate trade-offs that can be vital to the design of blood oxygenation sensitive imaging experiments for rCMRO2calculation.

CellSNAP: a fast, accurate algorithm for 3D cell segmentation in quantitative phase imaging.

Significance: Three-dimensional quantitative phase imaging (QPI) has rapidly emerged as a complementary tool to fluorescence imaging, as it provides an objective measure of cell morphology and dynamics, free of variability due to contrast agents. It has opened up new directions of investigation by providing systematic and correlative analysis of various cellular parameters without limitations of photobleaching and phototoxicity. While current QPI systems allow the rapid acquisition of tomographic images, the pipeline to analyze these raw three-dimensional (3D) tomograms is not well-developed. We focus on a critical, yet often underappreciated, step of the analysis pipeline that of 3D cell segmentation from the acquired tomograms. Aim: We report the CellSNAP (Cell Segmentation via Novel Algorithm for Phase Imaging) algorithm for the 3D segmentation of QPI images. Approach: The cell segmentation algorithm mimics the gemstone extraction process, initiating with a coarse 3D extrusion from a two-dimensional (2D) segmented mask to outline the cell structure. A 2D image is generated, and a segmentation algorithm identifies the boundary in the x-y plane. Leveraging cell continuity in consecutive z-stacks, a refined 3D segmentation, akin to fine chiseling in gemstone carving, completes the process. Results: The CellSNAP algorithm outstrips the current gold standard in terms of speed, robustness, and implementation, achieving cell segmentation under 2 s per cell on a single-core processor. The implementation of CellSNAP can easily be parallelized on a multi-core system for further speed improvements. For the cases where segmentation is possible with the existing standard method, our algorithm displays an average difference of 5% for dry mass and 8% for volume measurements. We also show that CellSNAP can handle challenging image datasets where cells are clumped and marred by interferogram drifts, which pose major difficulties for all QPI-focused AI-based segmentation tools. Conclusion: Our proposed method is less memory intensive and significantly faster than existing methods. The method can be easily implemented on a student laptop. Since the approach is rule-based, there is no need to collect a lot of imaging data and manually annotate them to perform machine learning based training of the model. We envision our work will lead to broader adoption of QPI imaging for high-throughput analysis, which has, in part, been stymied by a lack of suitable image segmentation tools.

ICG-Fluorescence Imaging for Margin Assessment During Minimally Invasive Colorectal Liver Metastasis Resection.

Importance: Unintended tumor-positive resection margins occur frequently during minimally invasive surgery for colorectal liver metastases and potentially negatively influence oncologic outcomes. Objective: To assess whether indocyanine green (ICG)-fluorescence-guided surgery is associated with achieving a higher radical resection rate in minimally invasive colorectal liver metastasis surgery and to assess the accuracy of ICG fluorescence for predicting the resection margin status. Design, Setting, and Participants: The MIMIC (Minimally Invasive, Indocyanine-Guided Metastasectomy in Patients With Colorectal Liver Metastases) trial was designed as a prospective single-arm multicenter cohort study in 8 Dutch liver surgery centers. Patients were scheduled to undergo minimally invasive (laparoscopic or robot-assisted) resections of colorectal liver metastases between September 1, 2018, and June 30, 2021. Exposures: All patients received a single intravenous bolus of 10 mg of ICG 24 hours prior to surgery. During surgery, ICG-fluorescence imaging was used as an adjunct to ultrasonography and regular laparoscopy to guide and assess the resection margin in real time. The ICG-fluorescence imaging was performed during and after liver parenchymal transection to enable real-time assessment of the tumor margin. Absence of ICG fluorescence was favorable both during transection and in the tumor bed directly after resection. Main Outcomes and Measures: The primary outcome measure was the radical (R0) resection rate, defined by the percentage of colorectal liver metastases resected with at least a 1 mm distance between the tumor and resection plane. Secondary outcomes were the accuracy of ICG fluorescence in detecting margin-positive (R1; <1 mm margin) resections and the change in surgical management. Results: In total, 225 patients were enrolled, of whom 201 (116 [57.7%] male; median age, 65 [IQR, 57-72] years) with 316 histologically proven colorectal liver metastases were included in the final analysis. The overall R0 resection rate was 92.4%. Re-resection of ICG-fluorescent tissue in the resection cavity was associated with a 5.0% increase in the R0 percentage (from 87.4% to 92.4%; P < .001). The sensitivity and specificity for real-time resection margin assessment were 60% and 90%, respectively (area under the receiver operating characteristic curve, 0.751; 95% CI, 0.668-0.833), with a positive predictive value of 54% and a negative predictive value of 92%. After training and proctoring of the first procedures, participating centers that were new to the technique had a comparable false-positive rate for predicting R1 resections during the first 10 procedures (odds ratio, 1.36; 95% CI, 0.44-4.24). The ICG-fluorescence imaging was associated with changes in intraoperative surgical management in 56 (27.9%) of the patients. Conclusions and Relevance: In this multicenter prospective cohort study, ICG-fluorescence imaging was associated with an increased rate of tumor margin-negative resection and changes in surgical management in more than one-quarter of the patients. The absence of ICG fluorescence during liver parenchymal transection predicted an R0 resection with 92% accuracy. These results suggest that use of ICG fluorescence may provide real-time feedback of the tumor margin and a higher rate of complete oncologic resection.

Two cases of laparoscopic deroofing of giant liver cysts using indocyanine green fluorescence imaging.

Laparoscopic deroofing (LD) for giant liver cysts using indocyanine green (ICG) fluorescence imaging was performed in two patients: a 53-year-old man with a 26-cm, symptomatic cyst and a 50-year-old woman with a 13-cm, symptomatic cyst. ICG fluorescence imaging can be used to easily identify the boundary between the liver parenchyma and the liver cyst. No postoperative bile leakage was observed in both patients. ICG fluorescence imaging is expected to become a desirable procedure in LD for giant liver cysts to reduce the occurrence of perioperative complications.

Widefield functional speckle-correlation optical scattering mesoscopy toward hemodynamic imaging.

Speckle-correlation optical scattering imaging (SCOSI) has shown the potential for non-invasive biomedical diagnostic applications, which directly utilizes the scattering patterns to reconstruct the deep and non-line-of-sight objects. However, the course of the translation of this technique to preclinical biomedical imaging applications has been postponed by the following two facts: 1) the field of view of SCOSI was significantly limited by the optical memory effect, and 2) the molecular-tagged functional imaging of the biological tissues remains largely unexplored. In this work, a proof-of-concept design of the first-generation widefield functional SCOSI (WF-SCOSI) system was presented for simultaneously achieving mesoscopic mapping of fluid morphology and flow rate, which was realized by implementing the concepts of scanning synthesis and fluorescence scattering flowmetry. The ex vivo imaging results of the fluorescence-labeled large-scale blood vessel network phantom underneath the strong scatters demonstrated the effectiveness of WF-SCOSI toward non-invasive hemodynamic imaging applications.

[Preliminary application of indocyanine green fluorescence imaging technology in nasal endoscopic tumor surgery].

Objective:To preliminarily study the practical value of Indocyanine green（ICG） molecular fluorescence imaging technology in nasal endoscopic tumor surgery. Methods:Five patients with tumors related to nasal sinuses, orbital wall and skull base in the Department of Otolaryngology head and Neck Surgery, General Hospital of Xinjiang Military Command from December 2022 to April 2023 were enrolled. Among them, 3 were benign tumors and 2 were malignant tumors. All patients underwent surgery under the guidance of ICG molecular fluorescence imaging. ICG was administered intravenously through cubital vein at a dose of 0.5 mg/kg 12 to 24 h before surgery. Tumors were labeled by fluorescence imaging during the operation. surgeons cleared the tumor tissue strictly according to the labeled range and depth, malignant tumors were further expanded and cleaned according to pathology results. Results:All 5 patients achieved accurate tumor localization with the aid of fluorescence imaging technology. Resections were performed with reference to fluorescent labeling boundaries, all patients achieved complete tumor cleanup or negative margins. Conclusion:For tumor-related surgery under nasal endoscopy, ICG molecular fluorescence imaging technology can not only achieve accurate real-time positioning, but also provide evidence for surgeons to judge tumor boundaries. Therefore, we believe that the technology should have certain practical value in nasal endoscopic tumor surgery.

Acto3D: an open-source user-friendly volume rendering software for high-resolution 3D fluorescence imaging in biology.

Advances in fluorescence microscopy and tissue-clearing have revolutionised 3D imaging of fluorescently labelled tissues, organs and embryos. However, the complexity and high cost of existing software and computing solutions limit their widespread adoption, especially by researchers with limited resources. Here, we present Acto3D, an open-source software, designed to streamline the generation and analysis of high-resolution 3D images of targets labelled with multiple fluorescent probes. Acto3D provides an intuitive interface for easy 3D data import and visualisation. Although Acto3D offers straightforward 3D viewing, it performs all computations explicitly, giving users detailed control over the displayed images. Leveraging an integrated graphics processing unit, Acto3D deploys all pixel data to system memory, reducing visualisation latency. This approach facilitates accurate image reconstruction and efficient data processing in 3D, eliminating the need for expensive high-performance computers and dedicated graphics processing units. We have also introduced a method for efficiently extracting lumen structures in 3D. We have validated Acto3D by imaging mouse embryonic structures and by performing 3D reconstruction of pharyngeal arch arteries while preserving fluorescence information. Acto3D is a cost-effective and efficient platform for biological research.

An efficient lipid droplet-targeted fluorescent probe for detection of intracellular viscosity.

Lipid droplet, an intracellular lipid reservoir, is vital for energy metabolism and signal transmission in cells. The viscosity directly affects the metabolism of lipid droplets, and the abnormal viscosity is associated with the occurrence and development of various diseases. Therefore, it is indispensable to develop techniques that can detect viscosity changes in intracellular lipid droplets. Based on twisted intramolecular charge transfer (TICT) mechanism, a novel small-molecule lipid droplet-targeted viscosity fluorescence probe PPF-1 was designed. The probe was easy to synthesize, it had a large Stokes shift, stable optical properties, and low bio-toxicity. Compared to being in methanol solution, the fluorescence intensity of PPF-1 in glycerol solution was increased 26.7-fold, and PPF-1 showed excellent ability to target lipid droplets. Thus, the probe PPF-1 could provide an effective means of detecting viscosity changes of lipid droplets and was of great value for physiological diagnosis of related diseases, pathological analysis, and medical research.

Novel activated NIR-II fluorescence/Ratio photoacoustic probe for dual-modality accurate imaging of palladium ions overload in mouse liver.

Palladium contaminants can pose risks to human health and the natural environment. Once Pd2+ enters the body, it can bind with DNA, proteins, and other macromolecules, disrupting cellular processes and causing serious harm to health. Therefore, it becomes critical to develop simple, highly selective and precise methods for detecting Pd2+in vivo. Here, we have successfully developed the first activated second near-infrared region fluorescence (NIR-II FL) and ratio photoacoustic (PA) probe NYR-1 for dual-modal accurate detection of Pd2+ levels. NYR-1 is capable of rapidly (< 60 s) and sensitively detection of Pd2+ in solution, providing switched on NIR-II FL920 and ratio PA808/PA720 dual-mode signal change. More notably, the probe NYR-1 was successfully used for non-invasive imaging of Pd2+ overload in mouse liver by NIR-II FL/Ratio PA dual-modality imaging technology for the first time. Thus, this work opens up a promising dual-modal detection method for the precise detection of Pd2+ in organisms and in the environment.

Imaging gastrointestinal damage due to acute mercury poisoning using a mitochondria-targeted dual near-infrared fluorescent probe.

Mercury (Hg) is one of the most widespread pollutants that pose serious threats to public health and the environment. People are inevitably exposed to Hg via different routes, such as respiration, dermal contact, drinking or diet. Hg poisoning could cause gingivitis, inflammation, vomiting and diarrhea, respiratory distress or even death. Especially during the developmental stage, there is considerable harm to the brain development of young children, causing serious symptoms such as intellectual disability and motor impairments, and delayed neural development. Therefore, it's of great significance to develop a specific, quick, practical and labor-saving assay for monitoring Hg2+. Herein, a mitochondria-targeted dual (excitation 700 nm and emission 728 nm) near-infrared (NIR) fluorescent probe JZ-1 was synthesized to detect Hg2+, which is a turn-on fluorescent probe designed based on the rhodamine fluorophore thiolactone, with advantages of swift response, great selectivity, and robust anti-interference capability. Cell fluorescence imaging results showed that JZ-1 could selectively target mitochondria in HeLa cells and monitor exogenous Hg2+. More importantly, JZ-1 has been successfully used to monitor gastrointestinal damage of acute mercury poisoning in a drug-induced mouse model, which provided a great method for sensing Hg species in living subjects, as well as for prenatal diagnosis.

Supermolecular Confined Silicon Phosphorescence Nanoprobes for Time-Resolved Hypoxic Imaging Analysis.

Room temperature phosphorescence (RTP) nanoprobes play crucial roles in hypoxia imaging due to their high signal-to-background ratio (SBR) in the time domain. However, synthesizing RTP probes in aqueous media with a small size and high quantum yield remains challenging for intracellular hypoxic imaging up to present. Herein, aqueous RTP nanoprobes consisting of naphthalene anhydride derivatives, cucurbit[7]uril (CB[7]), and organosilicon are reported via supermolecular confined methods. Benefiting from the noncovalent confinement of CB[7] and hydrolysis reactions of organosilicon, such small-sized RTP nanoprobes (5-10 nm) exhibit inherent tunable phosphorescence (from 400 to 680 nm) with microsecond second lifetimes (up to ∼158.7 µs) and high quantum yield (up to ∼30%). The as-prepared RTP nanoprobes illustrate excellent intracellular hypoxia responsibility in a broad range from ∼0.1 to 21% oxygen concentrations. Compared to traditional fluorescence mode, the SBR value (∼108.69) of microsecond-range time-resolved in vitro imaging is up to 2.26 times greater in severe hypoxia (<0.1% O2), offering opportunities for precision imaging analysis in a hypoxic environment.

Design and Screening of Fluorescent Probes Based upon Hemicyanine Dyes for Monitoring Mitochondrial Viscosity in Living Cells.

Viscosity, at the subcellular level, plays a crucial role as a physicochemical factor affecting microenvironment homeostasis. Abnormal changes in mitochondrial viscosity often lead to various diseases in the organism. Based on the twisted intramolecular charge transfer mechanism, four hemicyanine dye fluorescent probes (HT-SA, HT-SA-S, HT-Bzh, and HT-NA) were designed and synthesized for viscosity response. The single bond between the nitrogen-containing heterocycle and the carbon-carbon double in the structure of the probe bond served as the viscosity response site. Finally, the probe HT-Bzh was screened as the optimal mitochondrial viscosity probe according to its responsiveness, targeting, and interference resistance. The fluorescence intensity of the probe HT-Bzh increased 22-fold when the viscosity was increased from 13.75 to 811.2 cP. In summary, all four viscosity probes we have developed can be used in different applications depending on the external environment, providing a valuable reference for the design of potential tools to address viscosity monitoring in biological systems.

A ß-Galactosidase-Activated Fluorogenic Reporter for the Detection of Gastric Cancer In Vivo and in Urine.

Although gastric cancer (GC) is one of the most frequent malignant tumors in the digestive tract with high morbidity and mortality, it remains a diagnostic dilemma due to its reliance on invasive biopsy or insensitive assays. Herein, we report a fluorescent gastric cancer reporter (FGCR) with activatable near-infrared fluorescence (NIRF) signals and high renal-clearance efficiency for the detection of orthotopic GC in a murine model via real-time imaging and remote urinalysis. In the presence of gastric-tumor-associated ß-galactosidase (ß-Gal), FGCR can be fluorescently activated for in vivo NIRF imaging. Relying on its high renal-clearance efficiency (∼95% ID), it can be rapidly excreted through kidneys to urine for the ultrasensitive detection of tumors with a diameter down to ∼2.1 mm and for assessing the prognosis of oxaliplatin-based chemotherapy. This study not only provides a new approach for noninvasive auxiliary diagnosis and prognosis of GC but also provides guidelines for the development of fluorescence probes for cancer diagnosis.

Role of indocyanine green fluorescence imaging for evaluating blood supply in the gastric conduit via the substernal route after McKeown minimally invasive esophagectomy.

BACKGROUND: Anastomotic leakage (AL) is a determining factor of morbidity and mortality after esophagectomy. Adequate perfusion of the gastric conduit is crucial for AL prevention. This study aimed to determine whether intraoperative angiography using indocyanine green (ICG) fluorescence improves the incidence of AL after McKeown minimally invasive esophagectomy (MIE) with gastric conduit via the substernal route (SR). METHODS: This retrospective cohort study included 120 patients who underwent MIE with gastric conduit via SR for esophageal cancer between February 2019 and April 2023. Of 120 patients, 88 experienced intraoperative angiography using ICG (ICG group), and 32 patients experienced intraoperative angiography without ICG (no-ICG group). Baseline characteristics and operative outcomes, including AL as the main concern, were compared between the 2 groups. In addition, the outcomes among patients in the ICG group with different levels of fluorescence intensity were compared. RESULTS: The ICG and no-ICG groups were comparable in baseline characteristics and operative outcomes. There was no significant difference between the 2 groups regarding the rate of AL (31.0% vs 37.5%; P = .505), median dates of AL (9 vs 9 days; P = .810), and severity of AL (88.9%, 11.11%, and 0.0% vs 66.7%, 16.7%, and 16.7% for grades I, II, and III, respectively; P = .074). Patients in the ICG group with lower intensity of ICG had higher rates of leakage (24.6%, 39.3%, and 100% in levels I, II, and III of ICG intensity, respectively; P = .04). CONCLUSION: The use of ICG did not seem to reduce the rate of AL. However, abnormal intensity of ICG fluorescence was associated with a higher rate of AL, which implies a predictive potential.